A comparative study of zero-shot inference with large language models and supervised modeling in breast cancer pathology classification.

Although supervised machine learning is popular for information extraction from clinical notes, creating large, annotated datasets requires extensive domain expertise and is time-consuming. Meanwhile, large language models (LLMs) have demonstrated promising transfer learning capability. In this study, we explored whether recent LLMs can reduce the need for large-scale data annotations. We curated a manually labeled dataset of 769 breast cancer pathology reports, labeled with 13 categories, to compare zero-shot classification capability of the GPT-4 model and the GPT-3.5 model with supervised classification performance of three model architectures: random forests classifier, long short-term memory networks with attention (LSTM-Att), and the UCSF-BERT model. Across all 13 tasks, the GPT-4 model performed either significantly better than or as well as the best supervised model, the LSTM-Att model (average macro F1 score of 0.83 vs. 0.75). On tasks with a high imbalance between labels, the differences were more prominent. Frequent sources of GPT-4 errors included inferences from multiple samples and complex task design. On complex tasks where large annotated datasets cannot be easily collected, LLMs can reduce the burden of large-scale data labeling. However, if the use of LLMs is prohibitive, the use of simpler supervised models with large annotated datasets can provide comparable results. LLMs demonstrated the potential to speed up the execution of clinical NLP studies by reducing the need for curating large annotated datasets. This may increase the utilization of NLP-based variables and outcomes in observational clinical studies.

Predictive Modeling of Drug-Related Adverse Events with Real-World Data: A Case Study of Linezolid Hematologic Outcomes.

Electronic health records (EHRs) provide meaningful knowledge of drug-related adverse events (AEs) that are not captured in standard drug development and postmarketing surveillance. Using variables obtained from EHR data in the University of California San Francisco de-identified Clinical Data Warehouse, we aimed to evaluate the potential of machine learning to predict two hematological AEs, thrombocytopenia and anemia, in a cohort of patients treated with linezolid for 3 or more days. Features for model input were extracted at linezolid initiation (index), and outcomes were characterized from index to 14 days post-treatment. Random forest classification (RFC) was used for AE prediction, and reduced feature models were evaluated using cumulative importance (cImp) for feature selection. Grade 3+ thrombocytopenia and anemia occurred in 31% of 2,171 and 56% of 2,170 evaluable patients, respectively. Of the total 53 features, as few as 7 contributed at least 50% cImp, resulting in prediction accuracies of 70% or higher and area under the receiver operating characteristic curves of 0.886 for grade 3+ thrombocytopenia and 0.759 for grade 3+ anemia. Sensitivity analyses in strictly defined patient subgroups revealed similarly high predictive performance in full and reduced feature models. A logistic regression model with the same 50% cImp features showed similar predictive performance as RFC and good concordance with RFC probability predictions after isotonic calibration, adding interpretability. Collectively, this work demonstrates potential for machine learning prediction of AE risk in real-world patients using few variables regularly available in EHRs, which may aid in clinical decision making and/or monitoring.

Introducing a Comprehensive Score of Systemic Anticancer Treatment Relevance.

Treatment patterns in systemic anticancer therapy (SACT) are extremely varied and complex. While professional society guidelines exist that suggest recommended treatment strategies, these guidelines are produced through an extremely laborious and sometimes opaque manual process, making it impossible for such guidelines to cover all relevant treatment scenarios. To complement these manually curated guidelines, we leveraged a database of 5818 clinical trials and 7012 supporting references from 1943-present to calculate a quantifiable "relevance score". In a pilot evaluation, this score was strongly associated with professional society guideline recommendations, while also providing relevance information on thousands of additional therapies. We show that this score also accurately illustrates trends in SACT adoption over time. We foresee that this score, which comprehensively evaluates the relevance of SACT overall and by cancer subtype, will have utility for clinical practitioners as well as researchers in real-world data.

Topic modeling on clinical social work notes for exploring social determinants of health factors.

Objective: Existing research on social determinants of health (SDoH) predominantly focuses on physician notes and structured data within electronic medical records. This study posits that social work notes are an untapped, potentially rich source for SDoH information. We hypothesize that clinical notes recorded by social workers, whose role is to ameliorate social and economic factors, might provide a complementary information source of data on SDoH compared to physician notes, which primarily concentrate on medical diagnoses and treatments. We aimed to use word frequency analysis and topic modeling to identify prevalent terms and robust topics of discussion within a large cohort of social work notes including both outpatient and in-patient consultations. Materials and methods: We retrieved a diverse, deidentified corpus of 0.95 million clinical social work notes from 181 644 patients at the University of California, San Francisco. We conducted word frequency analysis related to ICD-10 chapters to identify prevalent terms within the notes. We then applied Latent Dirichlet Allocation (LDA) topic modeling analysis to characterize this corpus and identify potential topics of discussion, which was further stratified by note types and disease groups. Results: Word frequency analysis primarily identified medical-related terms associated with specific ICD10 chapters, though it also detected some subtle SDoH terms. In contrast, the LDA topic modeling analysis extracted 11 topics explicitly related to social determinants of health risk factors, such as financial status, abuse history, social support, risk of death, and mental health. The topic modeling approach effectively demonstrated variations between different types of social work notes and across patients with different types of diseases or conditions. Discussion: Our findings highlight LDA topic modeling's effectiveness in extracting SDoH-related themes and capturing variations in social work notes, demonstrating its potential for informing targeted interventions for at-risk populations. Conclusion: Social work notes offer a wealth of unique and valuable information on an individual's SDoH. These notes present consistent and meaningful topics of discussion that can be effectively analyzed and utilized to improve patient care and inform targeted interventions for at-risk populations.

Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study.

BACKGROUND: Large language models (LLMs) such as GPT-4 hold great promise as transformative tools in health care, ranging from automating administrative tasks to augmenting clinical decision making. However, these models also pose a danger of perpetuating biases and delivering incorrect medical diagnoses, which can have a direct, harmful impact on medical care. We aimed to assess whether GPT-4 encodes racial and gender biases that impact its use in health care. METHODS: Using the Azure OpenAI application interface, this model evaluation study tested whether GPT-4 encodes racial and gender biases and examined the impact of such biases on four potential applications of LLMs in the clinical domain-namely, medical education, diagnostic reasoning, clinical plan generation, and subjective patient assessment. We conducted experiments with prompts designed to resemble typical use of GPT-4 within clinical and medical education applications. We used clinical vignettes from NEJM Healer and from published research on implicit bias in health care. GPT-4 estimates of the demographic distribution of medical conditions were compared with true US prevalence estimates. Differential diagnosis and treatment planning were evaluated across demographic groups using standard statistical tests for significance between groups. FINDINGS: We found that GPT-4 did not appropriately model the demographic diversity of medical conditions, consistently producing clinical vignettes that stereotype demographic presentations. The differential diagnoses created by GPT-4 for standardised clinical vignettes were more likely to include diagnoses that stereotype certain races, ethnicities, and genders. Assessment and plans created by the model showed significant association between demographic attributes and recommendations for more expensive procedures as well as differences in patient perception. INTERPRETATION: Our findings highlight the urgent need for comprehensive and transparent bias assessments of LLM tools such as GPT-4 for intended use cases before they are integrated into clinical care. We discuss the potential sources of these biases and potential mitigation strategies before clinical implementation. FUNDING: Priscilla Chan and Mark Zuckerberg.

Defining incidence and complications of fibrolamellar liver cancer through tiered computational analysis of clinical data.

The incidence and biochemical consequences of rare tumor subtypes are often hard to study. Fibrolamellar liver cancer (FLC) is a rare malignancy affecting adolescents and young adults. To better characterize the incidence and biochemical consequences of this disease, we combined a comprehensive analysis of the electronic medical record and national payer data and found that FLC incidence is likely five to eight times higher than previous estimates. By employing unsupervised learning on clinical laboratory data from patients with hyperammonemia, we find that FLC-associated hyperammonemia mirrors metabolic dysregulation in urea cycle disorders. Our findings demonstrate that advanced computational analysis of rich clinical datasets can provide key clinical and biochemical insights into rare cancers.

A Clinical Reasoning-Encoded Case Library Developed through Natural Language Processing.

IMPORTANCE: Case reports that externalize expert diagnostic reasoning are utilized for clinical reasoning instruction but are difficult to search based on symptoms, final diagnosis, or differential diagnosis construction. Computational approaches that uncover how experienced diagnosticians analyze the medical information in a case as they formulate a differential diagnosis can guide educational uses of case reports. OBJECTIVE: To develop a "reasoning-encoded" case database for advanced clinical reasoning instruction by applying natural language processing (NLP), a sub-field of artificial intelligence, to a large case report library. DESIGN: We collected 2525 cases from the New England Journal of Medicine (NEJM) Clinical Pathological Conference (CPC) from 1965 to 2020 and used NLP to analyze the medical terminology in each case to derive unbiased (not prespecified) categories of analysis used by the clinical discussant. We then analyzed and mapped the degree of category overlap between cases. RESULTS: Our NLP algorithms identified clinically relevant categories that reflected the relationships between medical terms (which included symptoms, signs, test results, pathophysiology, and diagnoses). NLP extracted 43,291 symptoms across 2525 cases and physician-annotated 6532 diagnoses (both primary and related diagnoses). Our unsupervised learning computational approach identified 12 categories of medical terms that characterized the differential diagnosis discussions within individual cases. We used these categories to derive a measure of differential diagnosis similarity between cases and developed a website ( universeofcpc.com ) to allow visualization and exploration of 55 years of NEJM CPC case series. CONCLUSIONS: Applying NLP to curated instances of diagnostic reasoning can provide insight into how expert clinicians correlate and coordinate disease categories and processes when creating a differential diagnosis. Our reasoning-encoded CPC case database can be used by clinician-educators to design a case-based curriculum and by physicians to direct their lifelong learning efforts.

Structural variants shape driver combinations and outcomes in pediatric high-grade glioma.

We analyzed the contributions of structural variants (SVs) to gliomagenesis across 179 pediatric high-grade gliomas (pHGGs). The most recurrent SVs targeted MYC isoforms and receptor tyrosine kinases (RTKs), including an SV amplifying a MYC enhancer in 12% of diffuse midline gliomas (DMG), indicating an underappreciated role for MYC in pHGG. SV signature analysis revealed that tumors with simple signatures were TP53 wild type (TP53WT) but showed alterations in TP53 pathway members PPM1D and MDM4. Complex signatures were associated with direct aberrations in TP53, CDKN2A and RB1 early in tumor evolution and with later-occurring extrachromosomal amplicons. All pHGGs exhibited at least one simple-SV signature, but complex-SV signatures were primarily restricted to subsets of H3.3K27M DMGs and hemispheric pHGGs. Importantly, DMGs with complex-SV signatures were associated with shorter overall survival independent of histone mutation and TP53 status. These data provide insight into the impact of SVs on gliomagenesis and the mechanisms that shape them.

PPM1D mutations are oncogenic drivers of de novo diffuse midline glioma formation.

The role of PPM1D mutations in de novo gliomagenesis has not been systematically explored. Here we analyze whole genome sequences of 170 pediatric high-grade gliomas and find that truncating mutations in PPM1D that increase the stability of its phosphatase are clonal driver events in 11% of Diffuse Midline Gliomas (DMGs) and are enriched in primary pontine tumors. Through the development of DMG mouse models, we show that PPM1D mutations potentiate gliomagenesis and that PPM1D phosphatase activity is required for in vivo oncogenesis. Finally, we apply integrative phosphoproteomic and functional genomics assays and find that oncogenic effects of PPM1D truncation converge on regulators of cell cycle, DNA damage response, and p53 pathways, revealing therapeutic vulnerabilities including MDM2 inhibition.

CovidCounties is an interactive real time tracker of the COVID19 pandemic at the level of US counties.

Management of the COVID-19 pandemic has proven to be a significant challenge to policy makers. This is in large part due to uneven reporting and the absence of open-access visualization tools to present local trends and infer healthcare needs. Here we report the development of CovidCounties.org, an interactive web application that depicts daily disease trends at the level of US counties using time series plots and maps. This application is accompanied by a manually curated dataset that catalogs all major public policy actions made at the state-level, as well as technical validation of the primary data. Finally, the underlying code for the site is also provided as open source, enabling others to validate and learn from this work.

CovidCounties - an interactive, real-time tracker of the COVID-19 pandemic at the level of US counties.

Management of the COVID-19 pandemic has proven to be a significant challenge to policy makers. This is in large part due to uneven reporting and the absence of open-access visualization tools to present local trends and infer healthcare needs. Here we report the development of CovidCounties.org, an interactive web application that depicts daily disease trends at the level of US counties using time series plots and maps. This application is accompanied by a manually curated dataset that catalogs all major public policy actions made at the state-level, as well as technical validation of the primary data. Finally, the underlying code for the site is also provided as open source, enabling others to validate and learn from this work.

Copy-number and gene dependency analysis reveals partial copy loss of wild-type SF3B1 as a novel cancer vulnerability.

Genomic instability is a hallmark of human cancer, and results in widespread somatic copy number alterations. We used a genome-scale shRNA viability screen in human cancer cell lines to systematically identify genes that are essential in the context of particular copy-number alterations (copy-number associated gene dependencies). The most enriched class of copy-number associated gene dependencies was CYCLOPS (Copy-number alterations Yielding Cancer Liabilities Owing to Partial losS) genes, and spliceosome components were the most prevalent. One of these, the pre-mRNA splicing factor SF3B1, is also frequently mutated in cancer. We validated SF3B1 as a CYCLOPS gene and found that human cancer cells harboring partial SF3B1 copy-loss lack a reservoir of SF3b complex that protects cells with normal SF3B1 copy number from cell death upon partial SF3B1 suppression. These data provide a catalog of copy-number associated gene dependencies and identify partial copy-loss of wild-type SF3B1 as a novel, non-driver cancer gene dependency.

Genomic Heterogeneity and Exceptional Response to Dual Pathway Inhibition in Anaplastic Thyroid Cancer.

Purpose: Cancers may resist single-agent targeted therapies when the flux of cellular growth signals is shifted from one pathway to another. Blockade of multiple pathways may be necessary for effective inhibition of tumor growth. We document a case in which a patient with anaplastic thyroid carcinoma (ATC) failed to respond to either mTOR/PI3K or combined RAF/MEK inhibition but experienced a dramatic response when both drug regimens were combined.Experimental Design: Multi-region whole-exome sequencing of five diagnostic and four autopsy tumor biopsies was performed. Meta-analysis of DNA and RNA sequencing studies of ATC was performed.Results: Sequencing revealed truncal BRAF and PIK3CA mutations, which are known to activate the MAPK and PI3K/AKT pathways, respectively. Meta-analysis demonstrated 10.3% cooccurrence of MAPK and PI3K pathway alterations in ATC. These tumors display a separate transcriptional profile from other ATCs, consistent with a novel subgroup of ATC.Conclusions: BRAF and PIK3CA mutations define a distinct subset of ATC. Blockade of the MAPK and PI3K pathways appears necessary for tumor response in this subset of ATC. This identification of synergistic activity between targeted agents may inform clinical trial design in ATC. Clin Cancer Res; 23(9); 2367-73. ©2016 AACR.

Genomic evolution and chemoresistance in germ-cell tumours.

Germ-cell tumours (GCTs) are derived from germ cells and occur most frequently in the testes. GCTs are histologically heterogeneous and distinctly curable with chemotherapy. Gains of chromosome arm 12p and aneuploidy are nearly universal in GCTs, but specific somatic genomic features driving tumour initiation, chemosensitivity and progression are incompletely characterized. Here, using clinical whole-exome and transcriptome sequencing of precursor, primary (testicular and mediastinal) and chemoresistant metastatic human GCTs, we show that the primary somatic feature of GCTs is highly recurrent chromosome arm level amplifications and reciprocal deletions (reciprocal loss of heterozygosity), variations that are significantly enriched in GCTs compared to 19 other cancer types. These tumours also acquire KRAS mutations during the development from precursor to primary disease, and primary testicular GCTs (TGCTs) are uniformly wild type for TP53. In addition, by functional measurement of apoptotic signalling (BH3 profiling) of fresh tumour and adjacent tissue, we find that primary TGCTs have high mitochondrial priming that facilitates chemotherapy-induced apoptosis. Finally, by phylogenetic analysis of serial TGCTs that emerge with chemotherapy resistance, we show how TGCTs gain additional reciprocal loss of heterozygosity and that this is associated with loss of pluripotency markers (NANOG and POU5F1) in chemoresistant teratomas or transformed carcinomas. Our results demonstrate the distinct genomic features underlying the origins of this disease and associated with the chemosensitivity phenotype, as well as the rare progression to chemoresistance. These results identify the convergence of cancer genomics, mitochondrial priming and GCT evolution, and may provide insights into chemosensitivity and resistance in other cancers.

The genomic landscape and evolution of endometrial carcinoma progression and abdominopelvic metastasis.

Recent studies have detailed the genomic landscape of primary endometrial cancers, but the evolution of these cancers into metastases has not been characterized. We performed whole-exome sequencing of 98 tumor biopsies including complex atypical hyperplasias, primary tumors and paired abdominopelvic metastases to survey the evolutionary landscape of endometrial cancer. We expanded and reanalyzed The Cancer Genome Atlas (TCGA) data, identifying new recurrent alterations in primary tumors, including mutations in the estrogen receptor cofactor gene NRIP1 in 12% of patients. We found that likely driver events were present in both primary and metastatic tissue samples, with notable exceptions such as ARID1A mutations. Phylogenetic analyses indicated that the sampled metastases typically arose from a common ancestral subclone that was not detected in the primary tumor biopsy. These data demonstrate extensive genetic heterogeneity in endometrial cancers and relative homogeneity across metastatic sites.

Characterizing genomic alterations in cancer by complementary functional associations.

Systematic efforts to sequence the cancer genome have identified large numbers of mutations and copy number alterations in human cancers. However, elucidating the functional consequences of these variants, and their interactions to drive or maintain oncogenic states, remains a challenge in cancer research. We developed REVEALER, a computational method that identifies combinations of mutually exclusive genomic alterations correlated with functional phenotypes, such as the activation or gene dependency of oncogenic pathways or sensitivity to a drug treatment. We used REVEALER to uncover complementary genomic alterations associated with the transcriptional activation of ß-catenin and NRF2, MEK-inhibitor sensitivity, and KRAS dependency. REVEALER successfully identified both known and new associations, demonstrating the power of combining functional profiles with extensive characterization of genomic alterations in cancer genomes.

Systematic screening reveals a role for BRCA1 in the response to transcription-associated DNA damage.

BRCA1 is a breast and ovarian tumor suppressor. Given its numerous incompletely understood functions and the possibility that more exist, we performed complementary systematic screens in search of new BRCA1 protein-interacting partners. New BRCA1 functions and/or a better understanding of existing ones were sought. Among the new interacting proteins identified, genetic interactions were detected between BRCA1 and four of the interactors: TONSL, SETX, TCEANC, and TCEA2. Genetic interactions were also detected between BRCA1 and certain interactors of TONSL, including both members of the FACT complex. From these results, a new BRCA1 function in the response to transcription-associated DNA damage was detected. Specifically, new roles for BRCA1 in the restart of transcription after UV damage and in preventing or repairing damage caused by stabilized R loops were identified. These roles are likely carried out together with some of the newly identified interactors. This new function may be important in BRCA1 tumor suppression, since the expression of several interactors, including some of the above-noted transcription proteins, is repeatedly aberrant in both breast and ovarian cancers.

Pan-cancer genetic analysis identifies PARK2 as a master regulator of G1/S cyclins.

Coordinate control of different classes of cyclins is fundamentally important for cell cycle regulation and tumor suppression, yet the underlying mechanisms are incompletely understood. Here we show that the PARK2 tumor suppressor mediates this coordination. The PARK2 E3 ubiquitin ligase coordinately controls the stability of both cyclin D and cyclin E. Analysis of approximately 5,000 tumor genomes shows that PARK2 is a very frequently deleted gene in human cancer and uncovers a striking pattern of mutual exclusivity between PARK2 deletion and amplification of CCND1, CCNE1 or CDK4-implicating these genes in a common pathway. Inactivation of PARK2 results in the accumulation of cyclin D and acceleration of cell cycle progression. Furthermore, PARK2 is a component of a new class of cullin-RING-containing ubiquitin ligases targeting both cyclin D and cyclin E for degradation. Thus, PARK2 regulates cyclin-CDK complexes, as does the CDK inhibitor p16, but acts as a master regulator of the stability of G1/S cyclins.

Identification of a pan-cancer oncogenic microRNA superfamily anchored by a central core seed motif.

MicroRNAs modulate tumorigenesis through suppression of specific genes. As many tumour types rely on overlapping oncogenic pathways, a core set of microRNAs may exist, which consistently drives or suppresses tumorigenesis in many cancer types. Here we integrate The Cancer Genome Atlas (TCGA) pan-cancer data set with a microRNA target atlas composed of publicly available Argonaute Crosslinking Immunoprecipitation (AGO-CLIP) data to identify pan-tumour microRNA drivers of cancer. Through this analysis, we show a pan-cancer, coregulated oncogenic microRNA 'superfamily' consisting of the miR-17, miR-19, miR-130, miR-93, miR-18, miR-455 and miR-210 seed families, which cotargets critical tumour suppressors via a central GUGC core motif. We subsequently define mutations in microRNA target sites using the AGO-CLIP microRNA target atlas and TCGA exome-sequencing data. These combined analyses identify pan-cancer oncogenic cotargeting of the phosphoinositide 3-kinase, TGFß and p53 pathways by the miR-17-19-130 superfamily members.

Pan-cancer patterns of somatic copy number alteration.

Determining how somatic copy number alterations (SCNAs) promote cancer is an important goal. We characterized SCNA patterns in 4,934 cancers from The Cancer Genome Atlas Pan-Cancer data set. Whole-genome doubling, observed in 37% of cancers, was associated with higher rates of every other type of SCNA, TP53 mutations, CCNE1 amplifications and alterations of the PPP2R complex. SCNAs that were internal to chromosomes tended to be shorter than telomere-bounded SCNAs, suggesting different mechanisms underlying their generation. Significantly recurrent focal SCNAs were observed in 140 regions, including 102 without known oncogene or tumor suppressor gene targets and 50 with significantly mutated genes. Amplified regions without known oncogenes were enriched for genes involved in epigenetic regulation. When levels of genomic disruption were accounted for, 7% of region pairs were anticorrelated, and these regions tended to encompass genes whose proteins physically interact, suggesting related functions. These results provide insights into mechanisms of generation and functional consequences of cancer-related SCNAs.